Gesture detection in interspersed radar and network traffic signals

ABSTRACT

Techniques for performing gesture recognition with an electronic device are disclosed where the electronic device has a wireless communications capability using beamforming techniques and includes a plurality of millimeter wave antenna modules, each antenna module including at least one transmit antenna and at least one receive antenna, the antennas being operable in one or more frequency ranges greater than 20 GHz. Performing gesture recognition includes: simultaneous operation of the at least one transmit antenna and the at least one receive antenna so as to provide a radar capability; and detecting a presence and motion of a reflective object by analyzing magnitude and phase of signals received by the at least one receive antenna and resulting from reflection of signals transmitted by the transmit antenna and reflected by the reflective object.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/880,079, filed Jul. 29, 2019, entitled “Gesture Detection InInterspersed Radar and Network Traffic Signals” which is incorporatedherein by reference in its entirety.

TECHNICAL FIELD

This disclosure relates to gesture recognition techniques, and moreparticularly to using transmit and receive communication antennas of anelectronic device to provide a radar capability for gestureclassification and control.

DESCRIPTION OF THE RELATED TECHNOLOGY

Electronic devices such as smart phones, tablets, or “Internet ofThings” (IoT) devices and appliances can be made more functional byequipping them with sensors configured to support gesture recognition,such that the electronic device may be controlled without necessarilybeing in physical contact with a user. For example, gesture recognitionenables users to perform certain functions by swiping a hand, finger, orstylus proximate to but not necessarily in contact with the electronicdevice. Potential uses include: turning the device on/off, turning thevolume up/down, flipping a page, scrolling a page up/down, for example.Gesture recognition may be particularly useful when the device does nothave a touch screen or when touching the screen is inconvenient (e.g.wet hands).

In the absence of the presently disclosed techniques, touch or gesturerecognition sensors used in electronic devices are generally capacitivesensing, infra-red (IR) motion detectors, or cameras with videoprocessing. Capacitive sensing and IR detection require dedicatedhardware that are relatively bulky; video processing of camera imageryis a very inefficient method in terms of power consumption andcomputational requirements since it needs continuous monitoring andprocessing

Thus, improved gesture recognition techniques are desirable.

BRIEF DESCRIPTION OF THE DRAWINGS

Details of one or more implementations of the subject matter describedin this specification are set forth in this disclosure and theaccompanying drawings. Other features, aspects, and advantages willbecome apparent from a review of the disclosure. Note that the relativedimensions of the drawings and other diagrams of this disclosure may notbe drawn to scale. The sizes, thicknesses, arrangements, materials,etc., shown and described in this disclosure are made only by way ofexample and should not be construed as limiting. Like reference numbersand designations in the various drawings indicate like elements.

FIG. 1 illustrates an example of a radar arrangement, according to animplementation.

FIG. 2 illustrates an example of a radar system operation in accordancewith an implementation.

FIG. 3 illustrates time and “tap” domains for radar operation of a chipset compatible with IEEE 802.11ad and/or IEEE 802.11ay.

FIG. 4 illustrates an example plot of Channel Impulse Response (CIR) fora Golay scheme radar using the 802.11ay channel estimation field (CEF)as the transmitted waveform.

FIG. 5 illustrates an example radar hardware setup and aniterative-processing flow-chart that may be executed by an associatedhost electronic device, according to some implementations.

FIG. 6 illustrates an example of a recording of magnitude and phase of asingle correlation-tap of a single antenna for a target moving outwardand inward with respect to the radar arrangement.

FIG. 7 illustrates an example of a two finger gesture that may berecognized using the presently disclosed techniques.

FIG. 8 illustrates examples for the spectrum of the Golay correlation,according to an implementation.

FIG. 9 illustrates three types of motion that may be detected in someimplementations.

FIG. 10 illustrates an example of a radar arrangement for gesturerecognition in a 2D plane, according to an implementation.

FIG. 11 illustrates an example of an interferometer measurement for asingle pair of receiver elements, according to an implementation.

FIG. 12 illustrates an example of resulting tracking of general behaviorof the phase differences, according to an implementation.

FIG. 13 illustrates an example of a tracked path in 2D and the enclosedellipse generated for the case of linear movement, according to animplementation.

FIG. 14 illustrates an example of a tracked path in 2D and the enclosedellipse generated for the case of circular movement, according to animplementation.

FIGS. 15A-15D illustrate a side-by-side comparison between (1) adecimated millimeter wave radar signal for detecting a gesture and (2) acontinuous (non-decimated) millimeter wave radar signal for detectingthe same gesture. FIG. 15A shows a time-domain plot of the phase of thereceived radar signal for a continuous (non-decimated) millimeter wavesignal reflected off of the user's hand. FIG. 15B shows a time-domainplot of the phase of the received radar signal for a decimatedmillimeter wave signal reflected off of the user's hand during the samedouble-tap gesture. FIG. 15C is an example of a spectrogram and shows afrequency-domain plot of the phase of the received radar signal for acontinuous (non-decimated) millimeter wave signal reflected off of theuser's hand. FIG. 15D is an example of a spectrogram and shows afrequency-domain plot of the phase of the received radar signal for thedecimated millimeter wave signal reflected off of the user's hand.

FIG. 16 illustrates additional examples of spectrograms generated inaccordance with embodiments of the present disclosure.

FIG. 17 illustrates samples of positive and negative double tappinggestures accurately detected by the slope-estimation, time-domain basedtechnique described above.

FIG. 18 illustrates additional examples of “positive” double-tappingdetection using the time-domain technique (user performs double-tappinggesture).

FIG. 19 illustrates additional examples of “negative” double-tappingdetection using the same time-domain technique (user does not performany double-tapping gesture).

DETAILED DESCRIPTION

The following description is directed to certain implementations for thepurposes of describing the innovative aspects of this disclosure.However, a person having ordinary skill in the art will readilyrecognize that the teachings herein may be applied in a multitude ofdifferent ways. The described implementations may be implemented in anydevice, apparatus, or system that includes a millimeter bandcommunications capability. In addition, it is contemplated that thedescribed implementations may be included in or associated with avariety of electronic devices such as, but not limited to: mobiletelephones, multimedia Internet enabled cellular telephones, mobiletelevision receivers, wireless devices, smartphones, smart cards,wearable devices such as bracelets, armbands, wristbands, rings,headbands and patches, etc., Bluetooth® devices, personal dataassistants (PDAs), wireless electronic mail receivers, hand-held orportable computers, netbooks, notebooks, smartbooks, tablets, printers,copiers, scanners, facsimile devices, global positioning system (GPS)receivers/navigators, cameras, digital media players (such as MP3players), camcorders, game consoles, wrist watches, clocks, calculators,television monitors, flat panel displays, electronic reading devices(e.g., e-readers), mobile health devices, computer monitors, autodisplays (including odometer and speedometer displays, etc.), cockpitcontrols and/or displays, steering wheels, camera view displays (such asthe display of a rear view camera in a vehicle), electronic photographs,electronic billboards or signs, projectors, architectural structures,microwaves, refrigerators, stereo systems, cassette recorders orplayers, DVD players, CD players, VCRs, radios, portable memory chips,washers, dryers, washer/dryers, automated teller machines (ATMs),parking meters, packaging (such as in electromechanical systems (EMS)applications including microelectromechanical systems (MEMS)applications, as well as non-EMS applications), aesthetic structures(such as display of images on a piece of jewelry or clothing) and avariety of EMS devices. The teachings herein also may be used inapplications such as, but not limited to, electronic switching devices,radio frequency filters, sensors, accelerometers, gyroscopes,motion-sensing devices, magnetometers, inertial components for consumerelectronics, parts of consumer electronics products, varactors, liquidcrystal devices, electrophoretic devices, drive schemes, manufacturingprocesses and electronic test equipment. Thus, the teachings are notintended to be limited to the implementations depicted solely in theFigures, but instead have wide applicability as will be readily apparentto one having ordinary skill in the art.

Details of one or more implementations of the subject matter describedin this specification are set forth in this disclosure, which includesthe description and claims in this document and the accompanyingdrawings. Other features, aspects and advantages will become apparentfrom a review of the disclosure. Note that the relative dimensions ofthe drawings and other diagrams of this disclosure may not be drawn toscale. The sizes, thicknesses, arrangements, materials, etc., shown anddescribed in this disclosure are made only by way of example and shouldnot be construed as limiting.

The systems, methods and devices of the disclosure each have severalinnovative aspects, no single one of which is solely responsible for thedesirable attributes disclosed herein.

Gesture Detection with Millimeter Wave Radar

One innovative aspect of the subject matter described in this disclosurerelates to recognizing gestures proximate to an electronic device orappliance using RF components disposed on the device. The RF componentsinclude a plurality of mm wave antenna modules, each antenna moduleincluding at least one transmit and one receive antenna, wherein saidantennas are operable at a frequency of 20 GHz or higher. The RFcomponents may be operable in the 60 GHz band (approximately 57-64 GHz)and may be compatible with the IEEE 802.11ad and/or IEEE 802.11ay Wi-Fiprotocols. In accordance with the presently disclosed techniques a radarcapability is provided by simultaneously operating a transmitter RFchain and a receiver RF chain.

The IEEE 802.11ad/y Wi-Fi protocols relate to advanced wirelesscommunication networks operating in the unlicensed 60 GHz band. Theprotocols contemplate a substantial improvement for Wi-Fi communicationsin terms of both data rates and latencies compared to the Wi-Fiprotocols of the unlicensed 2.4 and 5 GHz bands. The 60 GHz band is anunlicensed 57-64 GHz frequency band, also known as mm Wave frequencies.

One of the major challenges in providing reliable wireless communicationnetworks in the 60 GHz band is the relatively heavy attenuation andshadowing observed for channels operating in that spectrum. As a result,communication systems in the 60 GHz band relay on sophisticatedbeamforming techniques. That is, systems at 60 GHz uses dedicatedsignal-processing algorithms with highly directional antenna arrays thatprovide communications over electronically maneuverabledirectional-beams between transmitters and receivers of the network.

An 802.11ad/y packet starts with a short training field, followed by achannel estimation field (CEF), packet header, PHY-payload and optionalfields for gain control and additional training. The CEF is composed ofGolay complementary sequences (in this case, 128 symbols long) used inestimating the channel response characteristics.

The present disclosure contemplate gesture recognition techniquescompatible with an 802.11ad/y networking chip set that cab be operatedwith several RF chains simultaneously. In particular, the chip set maybe operated with two RF chains, one for transmission and one as areceiver. These two RF chains may be operated simultaneously to provideradar capabilities. Gesture recognition may be accomplished by analysisof the Golay correlation outputs. Examples of gesture recognitioninclude but are not limited to (a) finger-based gesture recognition for“slider control”; (b) detection of two-finger relative motion; and (c)2-D gesture (e.g., vertical or circle motion) in space.

FIG. 1 illustrates an example of a radar arrangement, according to animplementation. The arrangement 100 may be based on a communicationsystem compatible with the IEEE 802.11ad and/or IEEE 802.11ay Wi-Fiprotocols. In the illustrated implementation, the arrangement 100includes a base-band chip 110 (denoted as M-chip) and two radiofrequency (RF) chips, denoted as R-chip 120(1) and R-chip 120(2). In theillustrated example, the M-chip 110 includes a multiplexer (Mux) 113,and a digital-to-analog converter (DAC) 111, and amplifier 112 coupledwith the Mux 113. The M-chip 110 also includes an amplifier 114, coupledwith Mux 113, amplifier 115 and analog-to-digital converter (ADC) 116.The mux 113 is coupled with the R-chip 120(1) and the R-chip 120(2). Insome implementations, the M-chip 110 may operate all the signal andmanagement processing required for communication and radar processing,including for example, generating and processing transmitted andreceived signals. In addition, the M-chip may control channel-accessprotocols and additional beam-configurations operations. The two RFchips, R-chip 120(1) and R-chip 120(2), may be similar or identical, one(R-chip 120(1)) being configured as a transmitter and the other (R-chip120(2)) as a receiver. In some implementations, each chip may includeand control up to 32 or more antenna elements and may include respectivefunctional elements (not illustrated) such as power amplifiers (PA), lownoise amplifiers (LNA), phase shifters as well as control units forbeamforming operations. In some implementations, it is contemplated thatan array of approximately 32 small (2.5 mm width) antenna elements maybe disposed on a single R-chip. Such a configuration, advantageously,can generate a narrow beam for both transmission and reception, andthereby mitigate relatively high free space path loss anticipated foroperation in the 60 GHz band.

Advantageously, both the M-chip 110 and the R-chips 120(1) and 120(2)may be fully compliant with an applicable 802.11 standard. In acommunication mode, a single RF chip may operate, in a time-divisionduplex (TDD) fashion, as both a receiver and transmitter. However, thepresent techniques contemplate obtaining radar functionality byperforming receiving and transmitting simultaneously, an operating modethat may be enabled by adding functionality to the M-chip.

FIG. 2 illustrates an example of a radar system operation in accordancewith an implementation. An electromagnetic wave is transmitted from atransmit (Tx) module (e.g., R-chip 120(1)) and reflected back from atarget object 201. Some of the reflected electromagnetic wave isreceived by a receive (Rx) module (e.g., R-chip 120(2)). The receivedsignal may be sampled for purposes of detecting the presence of thetarget object 201. By estimating the time-of-flight and theangle-of-arrival, the location and speed of the target object 201 may beestimated. As noted above, in some implementations the radar system mayinclude thirty two antenna elements at each of the transmitter moduleand at the receiver module. As a result, the radar system may beconfigured to detect multiple objects and also more accurately estimatethe direction of arrival. By controlling the direction of transmission,the radar system may be provided with even further improved spatialseparation resolution.

Performance of a radar system may be expressed by the followingequation:

$P_{r} = \frac{P_{t}G_{t}G_{r}\sigma c^{2}}{\left( {4\pi} \right)^{3}f^{2}R^{r}}$where P_(r) is the transmit power, G_(t) and G_(r) are the gains of thetransmitter and receiver, respectively, c is the speed of light, f isthe transmitted signal carrier frequency, R is the range to the target,σ is the radar cross-section (RCS) and r represents the reflection type.The RCS (σ) is a unit-less factor which may depend on the specifictarget type of interest. For example, metallic objects have higher RCSthan human tissue. The reflection type r is usually set with values inthe range 2-4. For examples, Snell-based reflection is usually set withr=2 where scattering-based reflections are set with r=4. This parameteris intended to describe the situation where sometimes most of the energycan be reflected back in the direction of the transmission.

Digital signal processing (DSP) of signals transmitted and received maybe performed. In some implementations DSP may be performed by logicwithin or associated with M-Chip 110, for example. In someimplementations, advantageously, Golay sequences used for variousobjectives in an IEEE 802.11ad and/or IEEE 802.11ay modem may be adaptedfor the presently disclosed radar applications. Thus, communicationdigital hardware already contemplated for communications protocols inaccordance with IEEE 802.11ad and/or IEEE 802.11ay may be duallypurposed. The DSP may also include decimation and interpolation filtersfor mitigation of out-of-band noise or interference using either or bothof the high bandwidth of 3.52 GHz channel bonding (CB2) or lowerbandwidth of 1.76 GHz channel bonding (CB1). Finally, DSP may provideaccurate timing of the signals reflected from a target (plot 210) anddistinguish the received reflected signals (peak 212) from receivedtransmitter leakage (i.e., mutual coupling) signals (peak 213). DSP mayfurther be used to correctly mitigate and synchronize interference.

Plot 210 depicts an example plot of channel impulse response (CIR) in dBas a function of time. For convenience, time is represented in terms of“taps” where taps are time domain samples of a single packet-correlationof a channel estimation. Advantageously, channel estimation is executedusing complementary Golay sequences as defined by the above mentioned802.11 standards. More particularly, the disclosed techniques mayprovide radar capabilities using a networking chip set that is fullycompatible with the above mentioned 802.11 standards by simultaneouslyoperating transmitter and receiver RF-chains. Where radar operation isadapted for gesture recognition, face detection, etc., the time samplesof a single Golay correlation may adopt a notion of distance, instead ofa notion of time. This is because the time in a perspective of a singlecorrelation corresponds to the wave traveling-time from the radar systemto the target and back. For channel bonding (CB) 1, each tap correspondsto about 8 cm and for CB2 to about 4 cm.

FIG. 3 illustrates time and “tap” domains for radar operation of a chipset compatible with IEEE 802.11ad and/or IEEE 802.11ay. The plot 310illustrates correlation outputs vs time for three consecutive packets,occurring, respectively, during time intervals 311, 312 and 313. Forclarity of illustration, each Golay correlation output is depicted ascontributing to 5 samples, denoted by vertical arrows, numbered 1-15. Ina tap notation, these signals are denoted Tap 1-Tap 5; the time signalfor Tap 1, plot 321, is made from samples 1, 6 and 11 of the Golaycorrelation output; the time signal for Tap 2, plot 322 is made fromsamples 2, 7 and 12; the time signal for Tap 3, plot 323, is made fromsamples 3, 8 and 13; the time signal for Tap 4, plot 324 is made fromsamples 4, 9 and 14; and the time signal for Tap 5, plot 325, is madefrom samples 5, 10 and 15. Thus, from an algorithmic perspective,correlation outputs vs time, plot 310, may be regarded as fivetime-signals, each corresponding to a different distance from the radarsystem. Alternatively or in addition, an observation signal and a vectorsignal may be modeled such that, in each time, a vector of observationsis provided for each tap. That is, in a time t, the observed samplesmake up the vector X_(t)=(X_(t) ⁽¹⁾, X_(t) ⁽²⁾, . . . , X_(t) ^((N)^(T)) ), where N_(T) is the number of taps, i.e., the number of samplesprovided for a single Golay correlation. Referring still to FIG. 3,where the time sampling interval is the time interval betweentransmitted packets, the sample vector X₁ is made from samples 1, 2, 3,4 and 5, the sample vector X₂ is made from samples 6, 7, 8, 9, and 10and the third sample vector X₃ is made from the samples 11, 12, 13, 14and 15.

As indicated above, the present techniques contemplate the use ofcomplementary Golay sequences for target detection. In contrast toconventional frequency modulation continuous wave (FMCW) techniques,also known as chirp or Linear Frequency Modulated (LFM), a Golay radarscheme provides near-zero side lobes (advantageous, particularly, formulti-target detection). Traditionally, sonar and radar designersavoided using compressed pulses schemes, such as those presentlydisclosed, because the performance of such schemes is poor in scenarioswhere a target maneuvers with a relatively high speed. But the presentinventors have appreciated that use of the Golay radar scheme may beadvantageous, at least for the intended use of gesture recognition wherethe target range and speed are each relatively small.

FIG. 4 illustrates an example plot of Channel Impulse Response (CIR) fora Golay scheme radar using the 802.11ay channel estimation field (CEF)as the transmitted waveform. For a stationary target (v=0 m/sec) the CIRhas zero side-lobes around the tap of the target. However, even forrelatively high-speed target of 100 m/sec (360 Km/H) the CIR is hardlyaffected and still maintains multi-target detection capability. Only forspeeds of about 800 m/sec and above, not to be taken into considerationfor human gesture recognition, does a noticeable side-lobe increaseappear. Thus, for practical gesture recognition using millimeter-waveradar, a scheme based on the 802.11ay CEF Golay sequence is expected tooutperform a radar scheme based on Frequency Modulation Continuous Wave(FMCW) signal waveforms.

FIG. 5 illustrates an example radar hardware setup and aniterative-processing flow-chart that may be executed by an associatedhost electronic device, according to some implementations. The radarhardware setup may be implemented in a host electronic device thatincludes a processor. The electronic device may include a wirelesscommunications capability using beamforming techniques and including aplurality of millimeter wave antenna modules, each antenna moduleincluding at least one transmit antenna and at least one receiveantenna, wherein said antennas are operable in one or more frequencyranges greater than 20 GHz. The processor may be configured to performgesture recognition with the electronic device by simultaneouslyoperating the at least one transmit antenna and the at least one receiveantenna so as to provide a radar capability and detecting a presence andmotion of a reflective object by analyzing magnitude and phase ofsignals received by the at least one receive antenna and resulting fromreflection of signals transmitted by the transmit antenna and reflectedby the reflective object.

In the illustrated example, a gesture recognition system arrangement 500includes an Rx antenna 531, coupled with a radar receiver 533. A Golaycorrelator 535, samples of outputs of which are stored in a buffer 537.A host processor 539 may be configured to execute process flow stepsrelated to gesture recognition and control of an electronic deviceresponsive to recognize gestures. In the illustrated example, the hostprocessor 539 may be configured to read, at block 542, data from theradar buffer 537. A recognized gesture may be obtained and output byprocessing, at block 544, currently read data from the buffer 537 (and,optionally, previously read data). Finally, the processor may beconfigured to execute a graphical user interface (GUI) operation, atblock 546, responsive to the recognized gesture.

The arrangement 500 may be adapted to recognize any number of specifictypes of gestures. For better understanding of the disclosed techniques,three specific examples of gesture types are described below: (a)finger-based gesture recognition for “slider control”; (b) detection oftwo-finger relative motion; and (c) 2-D gesture (e.g., vertical orcircle motion) in space.

Considering first finger-based gesture recognition appropriate forcontrol of a virtual “slider” (e.g., a volume control), the gesturerecognition may be based on observations taken from the output of areceiver compliant with the above-mentioned 802.11 standards.Advantageously, outputs from the receiver's Golay correlation (that maybe ordinarily available for channel estimation) are sampled. Forfinger-based gesture recognition, correlation output samples are takenthat correspond to distances in the range 0-40 cm from the Rx antenna.Such a range corresponds to the first 5 correlation taps for channelbonding CB1 and 8-10 correlation taps for CB2 (where each tapcorresponds to half the distance of a CB1 tap). Finger movements forslider control (relatively slow, fine motions) have been found to behighly observable in the phase domain of the received signal. At 60 GHz(5 mm wavelength), for example, a 1 mm target displacement representsapproximately 144° of phase shift because such displacement changes thetraveled wave round trip distance by 2 mm (40% of the wavelength).Accordingly, in some implementations, samples of the phase of themeasured Golay correlation outputs are collected where each sampledGolay correlation is a complex number indicative of travel distance ofthe transmitted waveform distance.

In an example implementation, the 802.11ad channel estimation packetsmay be set to operate at a rate of 2 msec apart, thereby providingsamples of the phase of the Golay correlation output at a rate of 500Hz. To increase signal to noise ratio (SNR), in some implementations, 16consecutive Golay pulses (a few micro-seconds apart) may be provided ateach 2 msec interval. These samples may be coherently summed up in orderto provide a substantial SNR increase per reading without significantlyincreasing the power consumption. More particularly, in the exampleimplementation, the following sum may be calculated at each 2 msecinterval:

$X = {\sum\limits_{t = 1}^{N_{p}}{A_{t}e^{j\;\phi_{t}}}}$where N_(p) is the number of consecutive, micro-seconds-apart, channelestimation packets (i.e., 16 in this example), and A_(t)e^(jω) ^(t) isthe Golay correlation output of the channel estimation packet t out ofthe N_(p) number of packets. Because human gesture movements arerelatively slow, such motion is not observable across the N_(p) pulses,and the summing procedure described above consistently achieved goodcoherent gain.

In every processing iteration (FIG. 5, block 544), a processing of abatch of N_(B) samples

X_(t₁), … , X_(t_(N_(B)))may take place, some of these are newly read samples and some aresamples from previous iterations, where X_(t) corresponds to samplessummed, as described above, at 2 msec intervals. Because signalamplitude changes very slowly and is sensitive to temperature changes,the present techniques may rely mainly on information carried by thephase, more particularly the N_(B) phase samples

∠X_(t₁), … , ∠ X_(t_(N_(B))).

Such phase samples may also exhibit significant noise, both as a resultof motion of the target that is unrelated to an actual gesture to berecognized and as a result of thermal noise. Such phase noise may betreated as corresponding to a multiplicative factor of the receivedsignal, similar to a fading in communication models. More particularly,in the present example, the correlation output, for a given tap, may bemodeled as X=αA·ejϕ+n, where A>0 is a real number, and corresponds toobservation magnitudes (proportional to a target reflection factor), ϕis the phase of the correlation output, n is the thermal noise and a isa complex-valued multiplicative noise factor.

FIG. 6 illustrates an example of a recording of magnitude and phase of asingle correlation-tap of a single antenna for a target moving outwardand inward with respect to the radar arrangement. More particularly, themagnitude (plot 610) and phase (plot 620 of the single antenna singlecorrelation-tap are illustrated in FIG. 6. During time period 621 thetarget is moving outwards from the radar arrangement; during time period622 the target is moving inwards toward the radar arrangement. Ingeneral, a steady slope (either decrease or increase) of the phase maybe observed in each interval that results from the Doppler effect of thetarget's movement.

In the example plot, events 630(1), 630(2), 630(3) and 630(4) arecircled and indicate occurrences of abnormal phase behavior that maycorrespond to a multiplicative noise factor that may be related totarget instability.

In some implementations, effects of phase noise may be mitigated by oneor both of: (1) applying a piece-wise linear fit, and (2) applying amedian filtering for the evaluated linear-slopes of the piece-wiselinear fitting results. The motivation for piece-wise linear fit followsfrom the rather steady increase and decrease behavior of the phase thatis observed in substantial hand motions as well as extremely subtlefinger movements. The median filtering may be advantageous for denoisingthe abnormal multiplicative noise-factor events. Even an extremely shortmedian window has been found to provide a very steady and clean phasefor gesture control.

In some implementations, a number N_(a) of phase samples may be chosenfor an evaluation of a single linear fit in the observed sequence. In anexample implementation, setting N_(a)=8 samples per slope was found toprovide a good use experience. During tests with real modules compliantwith 802.11ad, the inventors have found that 5-20 samples at a samplerate of 500-1000 samples per second are sufficient to detect even verydelicate slider movement gestures and provide a good user experience.

In an example implementation, for a particular iteration that includesN_(n) new samples,

X_(t₁), …, X_(t_(N_(n)))a batch or N_(B) samples may be chosen for processing, where

${N_{B} = {N_{a} \cdot \left\lceil \frac{N_{n}}{N_{a}} \right\rceil}},$and includes a quantity of past (previously logged) samples,N_(B)−N_(n). A piece-wise least square (LS) linear fit may be carriedfor every N_(a) samples, out of the

$\left\lceil \frac{N_{n}}{N_{a}} \right\rceil$groups filled in the current batch of N_(B) samples. For the N_(a)samples

∠X_(t₁), …, ∠X_(t_(N_(a))).the LS linear fit is given by X_(t) ¹=a·t+b where the slope a andconstant b are given by the following equations:

${{a = \frac{\sum\limits_{i = 1}^{N_{a}}{\left( {t_{i} - \overset{\_}{t}} \right)\left( {{\angle X_{t_{i}}} - \overset{\_}{\angle X}} \right)}}{\sum\limits_{i = 1}^{N_{a}}\left( {t_{i} - \overset{\_}{t}} \right)^{2}}};{{{and}b} = {\overset{\_}{\angle X} - {a \cdot t}}}},$where t and ∠X are, respectively, the mean time and the mean phase for agiven sample:

${\overset{¯}{t} = {\frac{1}{N_{a}}{\sum\limits_{i = 1}^{N_{a}}t_{i}}}};{\overset{\_}{\angle X} = {\frac{1}{N_{a}}{\sum\limits_{i = 1}^{N_{a}}\overset{\_}{\angle X}}}}$

Advantageously, the linear fit may be simplified taking into accountthat only the slope may be of interest, not the exact location of thelinear fit. As a result, the computations of the constant b may beeliminated. In addition, the slopes may be evaluated with respect toarbitrary time units, t₁=1; t₂=2; . . . T_(N) _(a) =N_(a). In someimplementations, therefore, a sequence of slopes {a_(k)} are filtered bya moving median filter of length N_(m) to provide a sequence of filteredslopes {s_(k)} where the k-th filtered sample s_(k) is given by thefollowing equation:s _(k) median(a _(k−(N) _(m) ₎₊₁ ,a _(k−(N) _(m) ₎₊₂ , . . . a _(k)).

The present inventors have found selecting N_(m)=5 to provide a gooduser experience.

For the presently described finger-based gesture recognition appropriatefor control of a virtual “slider”, an example control technique may bedescribed as follows:

$\begin{matrix}{{L_{t} = \begin{Bmatrix}{- L} & : & {{t - 1 + \alpha} \leq {- L}} \\L & : & {L \leq {L_{t - 1} + {\alpha s_{t}}}} \\{L_{t - 1} + {\alpha s_{t}}} & : & {otherwise}\end{Bmatrix}};} & {{Eq}(1)}\end{matrix}$where L_(t) is the slider-control level at time t, α is an attenuationfactor, s_(t) is the current linear fit slope and [−L, L] is the rangeof operation. The attenuation parameter α may be set optimize the userexperience for at least a majority of users. Testing has shown avariation in user preference, with some users preferring lower a whileothers found a higher a more satisfactory. In some implementations, anattenuation of about 0.2 was found to provide an appealinguser-experience for most users. The main tradeoff with respect to userexperience is between target instability and responsivity. The gain mustbe set high enough to support a speed of tracking so that the systemfeels responsive enough on the one hand while on the other hand providessolid appearance for the slider-level so it will not shake (targetinstability is controlled by lowering the value of α.

In some implementations, a further mechanism for enhancing the userexperience may be introduced, in which a quantized version of the slidercontrol level is presented. In the quantized version of the slidercontrol, a quantized slider-level L^(Q)ϵ{0, 1, . . . , Q} is computed inthe following manner. Each time a range L or −L is reached by L_(t), thequantized level L^(Q) is incremented or decremented and L_(t) is set tozero. The value L may be chosen so that (a) all target instabilitybehaviors is maintained within a range [−L, L] too small to beobservable by the user and (b) the quantized slider control tracks thetarget movements closely (i.e., the boundaries L and −L are reached fastenough), even in response to gentle user movements, so that thequantized slider-level move in a responsive manner.

It should be noted that the phase samples in a given iteration

X_(t₁), …X_(t_(N_(B)))may be provided by the Golay correlators for all the taps that areobserved in a current setting. As indicated above, for the case of CB2,8-10 taps are of interest. The taps for which processing should takeplace are the ones corresponding to the actual location of the finger orhand position, but this location will not generally be known and mayvary during operation. Thus, a methodology to decide which of the tapsamples to use in every iteration is desirable. In some implementations,the methodology includes one or more of the following techniques: (1)taking the angle corresponding to maximal strength tap; (2) showing anaverage tracker; (3) updating the tracker based on maximal move slope

The first technique may include inspecting the Golay-correlationmagnitudes

❘X_(t₁)❘, …❘X_(t_(N_(B)))❘.Each of these samples may be available for all N_(T) taps (i.e., X_(t)⁽¹⁾, . . . , X_(t) ^((N) ^(T) ⁾). The first technique may find the tapindex i* that maximizes the magnitude i*=argmax_(1≤i≤N) _(T) (|X_(t)^((i))|) and evaluate the slopes based on the phase samplescorresponding to magnitude-maximizing taps, using, for the t-th sample,∠X_(t) ^(i*).

Taking the strongest tap as described above is not necessarily optimalin every gesture application. In particular, in an instance of gentlefigure movements, it may be the case where the strongest tap is notreflected from the finger but from the palm of the hand, for example (orother large and firmly positioned reflective target). The secondtechnique serves to mitigate this problem by showing the user an averageslider position based on all taps of interest. A variation of thistechnique may be to use a quantized slider position where the trackingL_(t) to initiate the quantization shift is the average on all trackers(with respect to taps). The motivation for this variation is that thetracker that relates to the actual gesture will be the dominant part andtherefore the average will reflect the correct tap to follow. Othertrackers will either be still, or resembles a noisy random-walk behaviorwhich on average sums up to zero movement.

The third technique may use the tracker that corresponds to the maximalmovement. The second and third techniques, for a set of tracking valuesL_(t) ^((i)), 1≤i≤N_(T) for every tap of interest, may be compared asfollows. The second technique may use the average slider control level

$\overset{\_}{L_{T}} = {\frac{1}{N_{T}}{\sum\limits_{i = 1}^{N_{T}}L_{t}^{(i)}}}$whereas the third technique may update the slider control level found byEq (1) based on the phase s_(t) ^((i*)), wherei*=argmax_(1≤i≤N) _(T) (|s _(t) ^((i))|)

In some implementations, the second technique contemplates carrying outaveraging on the basis of slope, i.e., computing a single tracker withan average slope by, for example, substituting in Eq (1), for s_(t) anaverage slope s_(t) (where averaging takes place over the taps).

In the presently described use case of finger-based gesture recognitionfor slider control, an important aspect includes the detection of apresence of a finger, its entrance into and exit from a region in whichslider control is to be actuated (i.e., starting the gesture andfinishing the gesture). Advantageously, enabling of slider-controlshould be introduced so that the slider-control is enabled only when auser intention for moving the slider is detected. Instances in which theuser does not want the slider to move are mainly during the insertion ofthe finger to the position of slider-control and the removal. In someimplementations, distinguishing instances in which slider control isdesired from those in which slider control is not desired may beprovided by testing that the slopes do not pass a certain threshold.That is, the technique may enable the tracker update in Eq (1) only when|s_(t) ^((i))|<s_(TH) ∀ 1≤i≤N_(T), where S_(TH) is a predefinedthreshold to enable the slider. It will be appreciated that sometrade-off exists in setting an appropriate value for S_(TH). For exampleif S_(TH) is set at too high a value, the slider may be enabled inundesirable occasions. Contrariwise, if S_(TH) is set at too low avalue, there may be occurrences when a user gesture intended to triggerslider movement is not recognized.

In some implementations, detection of a target is advantageouslymaintained using observations of magnitude. More particularly, if

${{\max\limits_{1 \leq i \leq N_{T}}\left( {❘X_{t}^{(i)}❘} \right)} > M_{Th}},$then presence of a target is recognized and the tracking of Eq (1) maybe implemented. Otherwise, the tracking of Eq (1) may be disabled.

A further detection rule, that may be advantageously employed, relatesto variation of magnitudes instead of magnitude per se. For example,such a rule may determine whether

${{\max\limits_{1 \leq i \leq N_{T}}{{std}\left( {{❘X_{t_{1}}^{(i)}❘},{❘X_{t_{2}}^{(i)}❘}\ ,\ldots,{❘X_{t_{N_{S}}}^{(i)}❘}} \right)}} > M_{Th}^{S}},$where std is the standard deviation estimation based on N_(s) recentsamples and M_(Th) ^(S) is a predefined threshold. The present inventorshave found that the foregoing detection rule is highly efficient forhuman hands and fingers due to the natural vibrations of living targetobjects.

Techniques for recognition of a second specific type of gesture,specifically detection of two-finger relative motion, will now bedescribed. The example gesture relates to detection of switch orincrement like commands provided using parallel moment of the index andmiddle fingers. FIG. 7 illustrates an example of a two finger gesturethat may be recognized using the presently disclosed techniques. The twoextreme states of this movement with the arrows indicating the directionof movement following the current state. For example, in Detail A, theindex finger is in the upper position and the arrow indicates itsfollowing move is downwards while the middle finger is at the lowerposition and the arrow indicates its following movement is upwards; indetail B, the index finger is in the lower position and the arrowindicates its following move is upwards while the middle finger is atthe upper position and the arrow indicates its following movement isdownwards.

The example gesture may be considered as a rather “gentle” gesture sinceoverall entire movement of both fingers is about 2-3 cm in total and maybe performed at a rather moderate speed. It is an easy to performgesture that comes without any effort from the user perspective. Adetection of this gesture can be applied to on-off switch or to a switchposition, say a counter of a certain state. In some implementations thegesture detection may be executed simultaneous to the slider controlfunctions described above so as to provide an overall operation of avirtual sound player device. For example, the two finger movement may beused to switch the sound track currently played and the single fingermovement may be used for volume control.

In some implementations, the detection of the two finger movement isbased on spectral analysis. Spectral analysis may be carried for acertain tap of interest sampled at the output of the Golay correlation.Each finger movement introduces a complex exponent with frequency offsetΔf corresponding to its speed. When two fingers are movingsimultaneously, the spectral analysis prominent may be expected toexhibit energy both in positive and negative frequencies. Where S_(f)denote the spectrum of a sample at frequency f, the following detectionrule may be applied:

$\begin{matrix}{{{\sum\limits_{f \in {\mathbb{F}}^{+}}1_{{❘S_{f}❘} \geq S^{th}}} \geq {N_{+}^{th}{AND}{\sum\limits_{f \in {\mathbb{F}}^{-}}1_{{❘S_{f}❘} \geq S^{th}}}} \geq N_{-}^{th}};} & {{Eq}(2)}\end{matrix}$where

⁺ and

⁻ are the sets of positive and negative frequency bins for spectralanalysis, |S_(f)| is the spectral density at frequency bin f, S^(th) isa threshold for prominent energy content at a given spectral bin, and N₊^(th) and N⁻ ^(th) are thresholds for the minimal number of frequencybins, in positive and negative frequencies, that are required to bestrong enough so that two finger movement is detected. FIG. 8illustrates examples for the spectrum of the Golay correlation,according to an implementation. Detail C relates to a single unmovingtarget (finger); Detail D relates to a single moving target (finger);and Detail E relates to two moving targets (fingers).

In some implementations, the sets

⁺ and

⁻ may be chosen so that detection is based on frequencies above acertain threshold. For example,

⁺ and

⁻ may be chosen such that

⁺={f>f^(th):fϵ

^(S)} and

⁺={f<f^(th):fϵ

^(S)}, where

^(S) is the set of all frequency bins available for spectral analysis(as defined by the time length of the analyzed interval at the samplingrate) and f^(th)>0 is a positive threshold. The above-mentioned choiceof

⁺ and

⁻ may be advantageous in view of the fact that a strong prominent energyaround DC (0 frequency) is generally exhibited due to the presence of astrong target (e.g., the palm of the user's hand).

In some implementations, reliability of the two finger movement detectormay be increased by discarding spectral analysis when highly fast andstrong in or out movement is detected based on an instantaneous phase ofthe signal. For example, where the current spectral analysis is based onN_(B) samples, X_(t) ₁ , X_(t) ₂ , . . . ,

X_(t_(N_(B))),the spectral analysis may be discarded when ∠X_(t) _(i) >α_(th), where1≤i≤N_(B), and α_(th) is a predefined threshold.

Alternatively or in addition, similar protection can be provided byinspecting the phase slopes or filtered slopes and, discarding thespectral analysis when some slope evaluated during a spectral intervalof interest exceeds a predetermined threshold.

Moreover, in some implementations, a further performance improvement maybe gained when spectral analysis is discarded for a few iterations afteran initial discard event takes place as a result of the above describedmethodology.

Yet another discarding rule may be based on the spectral analysisitself. For example, some strong movements unrelated to a two fingergesture have a strong spectral energy in either positive or negativebands and may be discarded by modifying the analysis of Eq (2). Themodifications may include replacing the logical AND with a logical OR,setting substantially higher threshold frequencies for

⁺ and

⁻, and setting bin counts N₊ ^(th) and N⁻ ^(th) to substantially higherlevels.

In some implementations, a further increase in detection reliability maybe obtained by looking for consecutive repetition of detection. Forexample, spectral analysis may be conducted in moving windows of time.When a consecutive quantity of consecutive detection rules agree inpositive detection for two finger movement, then the detection may beset positive.

Techniques for recognition of a third specific type of gesture,specifically the recognition of gestures in a 2D plane based on movementof a target object in a region within range of the radar arrangement,will now be described. FIG. 9 illustrates three types of motion thatwill be considered in some implementations. In particular, Detail F andG illustrate, respectively, linear motion of target object 201 in thehorizontal and vertical direction, and Detail H illustrates circularmovement of target object 201 in 2D space within range of radararrangement 900. FIG. 10 illustrates an example of a radar arrangementfor gesture recognition in a 2D plane, according to an implementation.In the illustrated example, a radar arrangement 1000 includes a singleelement transmit antenna and a three element array 1031 for the receiveantenna. Transmission and reception may be carried out simultaneously.In order to have a single reception chain at the receiver, in someimplementations, we receive each of the received channel estimationpackets in a different element in a consecutive order. In otherimplementations, in order to improve signal-to-noise ratio, severalconsecutive packets may be received at a single receive element beforeswitching to the next element. The received observations for eachelement can be coherently combined to increase the signal-to-noise ratioprovided that packets are transmitted fast enough.

In some implementations, a gesture recognition algorithm is based oninterferometer measurements in pairs. FIG. 11 illustrates an example ofan interferometer measurement for a single pair of receiver elements,according to an implementation. An estimate of the angle of arrival isprovided based on the phase difference of radiated signals reflectedfrom target object 201, and received by Rx elements 1031 a and 1031 b. Aclosed form expression for the angle of arrival may be derived usingtechniques analogous to those used in the direction finding andradio-astronomy disciplines.

Referring again to FIG. 10, for the illustrated example implementation,we may obtain phase difference observations for two antenna elementpairs, horizontal element pair (b, c) and a vertical element pair (a,b). For a specific gesture, it may be unnecessary to compute the exactangle of arrival, but instead obtain only the measurements of phasedifferences for the purposes of gesture recognition.

In some implementations, an algorithm provides an estimate to track thetarget object in the 2D space by applying Eq (1) where s_(t) isredefined as the slope of a linear fit of the phase differences and thetracking is carried out in parallel for both the horizontal and verticaldifferences. Linear fit and median filtering may be applied for thephase difference slopes to mitigate target-instability. FIG. 12illustrates an example of resulting tracking of general behavior of thephase differences, according to an implementation. The illustrated plotsresulted from operation of an 802.11 ad/y standard-compatible networkingchip set that operated with simultaneous receiving and transmitting RFchains for radar capabilities. Detail J shows the magnitude of signalreceived by one of the receiving antenna whereas Detail K shows thephase difference between the two receiving antennas are shown afterperforming a piece-wise linear-fit of median-filtered slopes. In each ofthe piece-wise linear-fit operations, the slope of the phase-differencethat corresponds to the tap having the strongest magnitude (Detail J)was selected where magnitude is measured in one or both of the receivingantenna modules. It is noted that if accurate tracking is of interest,than exact angle of arrival may be calculated and provided to a moresophisticated, classic or modern tracking algorithms. However, this isnot generally required for the present classification procedure.

In an example implementation, the classification algorithm may be basedon computing the minimal enclosing ellipse for an interval of estimatedtracked path in the 2D plane. Then the gesture may be classified basedon a ratio of the ellipse axes. More specifically, where the minimalenclosing ellipse is given in an (x, y) plane by

${{\frac{\left( {x - c_{1}} \right)^{2}}{a^{2}} + \frac{\left( {y - c_{2}} \right)^{2}}{b^{2}}} = 1},$then the classification may be based on the ratio

${E_{f} = \frac{\max\left( {a,b} \right)}{\min\left( {a,b} \right)}}.$

In the case of linear movement, there may be a substantial ratio betweenthe axes, whereas, in the case of a circular gesture, the axes aresimilar. The inventors have found that circular movement may be reliablyidentified when E_(f)≤2 while linear movement may be reliably identifiedwhen E_(f)≥3. The foregoing simple rule has been found to capture arather broad spectrum of linear and circular shapes to be counted aslinear and circular while providing enough separation between thegestures to give reliable classification. The inventors have also foundthat user experience is well maintained even if the minimal enclosingellipse is solved rather loosely (for purpose of simplifying/speedingcomputations). FIG. 13 illustrates an example of a tracked path in 2Dand the enclosed ellipse generated for the case of linear movement,according to an implementation. FIG. 14 illustrates an example of atracked path in 2D and the enclosed ellipse generated for the case ofcircular movement, according to an implementation. It may observed thatthe generated ellipse at hand is not enclosing the entire track ofmovement, this is a result of a rather loose solution for theoptimization problem at hand.

Gesture Detection in Interspersed Radar and Network Traffic Signals

Particularly useful embodiments of the disclosure involve gesturedetection in interspersed radar and network traffic signals. It may beadvantageous to implement both gesture detection and communication ofnetwork traffic using millimeter wave technology. According to thepresent disclosure, the same transmit and receive hardware used formillimeter wave network data communications may be be re-used formillimeter wave gesture detection. Techniques are employed to ensurethat millimeter wave gesture detection can be reliably achieved, e.g.,sufficient signal to noise ratio (SNR) realized, even in the presence ofmillimeter wave network traffic.

One illustrative gesture is a “double-tap” gesture, in which two fingersof a user's hand are used to tap an electronic device or part of anelectronic device, such as a screen on a mobile phone, in quicksuccession. While the double-tap gesture is illustrated below as anexample, the same or similar techniques can be applied to detection ofother types of gestures.

In one implementation, a test system is used to record both positive andnegative samples (instances) of the double-tap gesture. The test systemmay be one illustrated by FIGS. 1 and 5, for example. In one realisticexperiment, a user's hand is positioned approximately 30 cm from themillimeter wave radar transmitter and receiver, to perform both positiveand negative samples. For each positive sample, the user performs anactual double-tap gesture while the test system performs a millimeterwave gesture detection, using techniques such as those described herein.Either a single double-tapping gesture or two consecutive double-tappinggestures may be performed by the user. The test system may record thecaptured gesture detection data in memory. For each negative sample, auser performs a gesture that is not a double-tap gesture, e.g., ageneral hand, palm, and/or finger movement, while the test systemperforms a millimeter wave gesture detection. Again, the test system mayrecord the captured gesture detection data in memory. Here, memory maycomprise, for example, buffer 537 in FIG. 5. The experiment may besummarized as follows:

-   -   Gesture recorded approximately 30 cm facing the radar TX and RX        antennas    -   Double-tapping (Positive Samples)    -   General hand/palm/finger movements (Negative Samples)    -   Either a single or two events of double-tapping captured within        the recording    -   Hand is present during all recording duration

Millimeter wave gesture detection may be based on burst radar signals.In one illustrative example, a burst is transmitted from the TX antennaevery 1 msec. Each burst comprises 32 pulses, with pulses 10microseconds apart, for a burst length of 0.3 msec. In one simpleexample, a single antenna configuration is used. For antennas havingmultiple elements, a particular antenna element may be used while otherantenna elements may be ignored. For example, TX antenna element #16 andRX antenna element #16 may be used during all bursts and pulses. Outputfrom certain taps may be recorded. For example, taps [17-19] may berecorded, while taps 6-8 corresponding to OTA leakage may be ignored. Inthis particular example, 5 seconds of gesture data may be recorded. Thisrecorded data may comprise 5000 channel impulse responses (CIRs). Asmentioned previously, each CIR may correspond to a burst. Thus, 5000bursts, each having a 1 msec duration, may result in 5000 CIRs. Such anexample is summarized below:

-   -   A burst every 1 msec    -   Each burst=32 pulses (10 usec apart, total 0.3 ms)    -   Single antenna configuration: TX antenna element #16 and RX        antenna element #16 during all bursts and pulses (no BF used)    -   Taps [17-19] are recorded (Taps 6-8 from OTA leakage)    -   5 seconds are recorded (5000*32 CIRs)

Pre-processing may be performed to effectively increase SNR. Onetechnique that may be employed is by summing up all the CIRs from asingle burst to produce a single combined CIR. In the above example, 32CIRs from a single burst may be combined to generate a single CIR. Thisprovides a 15 dB SNR increase. Here, 10 microseconds is well within thecoherence time of the target. Also, in this example, computation isfurther simplified by assuming that no leakage cancellation is carried(DC bin is removed). Furthermore, received radar signal is only takenfrom a single tap, e.g., tap #17, for sake of simplicity. It is notedthat signals taken from adjacent taps were generally observed to be justas good. Also, no beamforming is used, again for sake of simplicity.Finally, two different data sets are processed, to compare theperformance of (1) millimeter wave radar gesture detection radar in thepresence of interspersed millimeter wave network traffic (“decimated”signal) vs. (2) continuous millimeter wave radar gesture detection.These conditions are summarized below:

-   -   32 CIRs form each burst are summed to provide a single CIR    -   No leakage cancellation is carried (DC bin is removed)    -   Only tap #17 is taken for analysis (adjacent taps were just as        good)    -   No beamforming is used (single antenna)    -   Two data sets:        -   (1) Decimated recording for 8 msec burst with 50% duty cycle            (8 msec radar, 8 msec network traffic)        -   (2) Continuous recording with CIR per 1 msec

As will be shown in later sections, even though the test system isgreatly simplified, results clearly show effective millimeter wave radargesture detection radar in the presence of interspersed millimeter wavenetwork traffic. Additional enhancements to performance may be achievedby employing more complicated features, such as employing beamforming,multiple taps, etc.

FIGS. 15A-15D illustrate a side-by-side comparison between (1) adecimated millimeter wave radar signal for detecting a gesture and (2) acontinuous (non-decimated) millimeter wave radar signal for detectingthe same gesture. FIG. 15A shows a time-domain plot of the phase of thereceived radar signal for a continuous (non-decimated) millimeter wavesignal reflected off of the user's hand. Here, the x-axis depicts timein units of msec, and the y-axis depicts phase in units of radians. Abox is draw over a duration of approximately 2000 msec, in which a“double-tap” gesture is being performed. As can be seen, the phase ofthe receive signal experiences sharp positive and negative values (e.g.,amplitudes peaking at +π and −π radians), indicating rapid in-outmovements associated with the double-tap gesture. FIG. 15B shows atime-domain plot of the phase of the received radar signal for adecimated millimeter wave signal reflected off of the user's hand duringthe same double-tap gesture. For example, this signal may comprise 8msec of continuous recording (i.e., millimeter wave radar burst),followed by 8 msec with no recording (i.e., millimeter wave networktraffic), which corresponds to a 50% duty cycle signal. It is worthwhileto note that the recorded signal is extremely prominent, and the gestureis clearly observed in both the continuous version and the decimatedversion of the received millimeter wave radar signal, indicating thefeasibility of using the decimated radar signal, which supportsinterspersed network traffic, to support gesture detection. FIG. 15C isan example of a spectrogram and shows a frequency-domain plot of thephase of the received radar signal for a continuous (non-decimated)millimeter wave signal reflected off of the user's hand. FIG. 15C is afrequency-domain equivalent of the time-domain plot shown in FIG. 15A.Here, the x-axis depicts time in units of msec, and the y-axis depictsspeed in units of meters per second (m/sec). FIG. 15D is an example of aspectrogram and shows a frequency-domain plot of the phase of thereceived radar signal for the decimated millimeter wave signal reflectedoff of the user's hand. FIG. 15D is a frequency-domain equivalent of thetime-domain plot shown in FIG. 15B. The spectrograms shown in FIGS. 15Cand 15D may be obtained by performing a transform on the time-domainradar signal. In the examples shown, a 64-length Fast Fourier Transform(FFT) with no windowing function is used. As shown in both FIGS. 15C and15D, the strong positive and negative Doppler frequencies appear vividlyin both the continuous (original) record (FIG. 15C) and the decimatedversion (FIG. 15D) of the spectrogram.

According to certain embodiments, gesture detection may be implementedusing a spectrogram approach. In the example described above, thenon-decimated received time-domain radar signal is received in Nsamples, taken 1 msec apart. This time-domain signal is converted to asignal-domain signal by applying a transform, in this case a 64-lengthFFT with no windowing. The samples are t, t−1, . . . , 5-63, for t=64:N.The decimated received time-domain signal, again, represents a 8 msecburst with 50% duty cycle (8 msec radar, 8 msec network traffic). Here,zero filling is employed to fill in the missing time-domain samples,then the same length-64 FFT (with no windowing) is applied. This resultsin an “on-off” modulation effect for the spectrum. However, the “on-off”modulation effect has no practical impact on the gesture spectralcharacterization. These details of the example are summarized below:

-   -   Non-decimated signal:        -   N samples, 1 msec apart        -   Take 64-length FFT (no window) for samples: t, t−1, . . . ,            t−63, for t=64:N    -   Decimated signal for 8 msec burst with 50% duty cycle (8 msec        radar, 8 msec network traffic):        -   For spectrogram we take an approach of filling in zeros for            the missing samples and then applying the same FFT            computation        -   This approach adds “on-off” modulation effect on the            spectrum

According to at least one embodiment of the disclosure, a machinelearning (ML) technique based on the spectrograms is used for gesturedetection. The ML classifier may be trained using spectrogram known tobe associated with the targeted gesture—i.e., positive spectrograms, aswell as spectrogram known to be associated with lack of the targetedgesture—i.e., negative spectrograms. Positive spectrograms may beobtained from radar signals received while a user's hand is performing adouble-tap gesture, for example. Negative spectrograms may be obtainedfrom radar signals received while a user's hand is performing generalhand/palm/finger movements that are not double-tap gestures. Spectrogramdata may be compressed using a compression technique, such as principlecomponent analysis (PCA) to reduce dimensions. This can reduce storageand computational requirements. Also, different types of ML classifiersmay be used, such as a bagged-tree classifier. The simplifiedmachine-learning, spectrogram-based gesture detection techniquedescribed here yielded 96% accuracy. Performance may be further improvedby using a greater number of samples in ML classifier training, etc.Details of the example machine learning classifier is summarized below.

-   -   Machine-learning classifier based on spectrogram images        -   25 positive and 25 negative spectrograms, each 500 msec        -   All data is processed via PCA to reduce dimensions        -   Ensemble bagged-tree classification showed results with 96%            accuracy        -   Process is repeated for full samples and for decimated            samples

FIG. 16 illustrates additional examples of spectrograms generated inaccordance with embodiments of the present disclosure. Note that fourspectrograms labeled as “Repeated double tapping” are spectrograms whichrecorded of the user performing the double tapping gesture twice, inquick succession. These are positive samples. The twelve priorspectrograms (not labeled) are spectrograms recorded of the userperforming the double tapping gesture once. These are also positivesamples. The six spectrograms labeled as “no-tapping recordings” arespectrograms recorded of the user not performing any double-tappinggesture. Here, the user's hand may remain present and may not be stillbut instead is performing delicate movements.

According to certain embodiments, gesture detection may be implementedusing an approach based on slope-estimation of the phase signalperformed in the time domain. Just as an example, input to such atime-domain gesture detector may be in the form of sampled RX signalfrom the I and Q channels of a quadrature demodulator, obtained as asequence of 500 samples spaced 1 msec apart. The output of thetime-domain gesture detector may be positive or negative identificationof a double-tapping gesture. The processing may involve first estimatinga sequence of slopes of the phase of received signal. This may involvedividing the received signal into slope estimation intervals, which maybe consecutive but disjoint intervals (e.g., 8 msec). Next, an unwrapoperation maybe performed on the sequence of estimated slopes, togenerate an unwrapped version of estimated slopes. Next, a linear fitoperation may be performed to obtain a slope estimate for each slopeestimation interval. A threshold may be established for determiningpositive versus negative slope. Using the threshold, the sequence ofslope estimates maybe converted into slope polarities, each beingpositive (“+”), negative (“−”), or zero (“0”). Next, the sequence ofslope polarities is checked for particular patterns, such as “+−+−” or“−+−+” and if such a pattern (with possible gaps within the providedsignal, e.g., +0−+00−, up to reasonable interval) appears, then apositive gesture detection decision is reported. Otherwise a negativedecision gesture detection decision is reported. This example of aslope-estimation gesture detector is summarized below:

-   -   Input: IQ samples of 0.5 sec, {y_(t)}_(t=1) ^(N), 500 samples 1        msec apart    -   Output: double-tapping identification    -   Processing:    -   Take the signal phase, {θ_(t)=angel(y_(t))}_(t=1) ^(N)    -   Divide the signal to slope estimation intervals, consecutive but        disjoint (8 msec), {(θ_(8(t−1)+1), θ_(8(t−1)+1), . . .        θ_(8(t−1)+8))}_(t=1) ^(N/8)    -   Unwrap the phase for each interval {(θ′_(8(t−1)+1),        θ′_(8(t−1)+1), . . . , θ′_(8(t−1)+8))=unwrap(θ_(8(t−1)+1),        θ_(8(t−1)+1), . . . , θ_(8(t−1)+8))}_(t=1) ^(N/8)    -   Compute linear fit slope for the unwrapped phase at each        interval {a_(t)=slope((θ′_(8(t−1)+1), θ′_(8(t−1)+1), . . .        θ′_(8(t−1)+8)))}_(t=1) ^(N/8)    -   Check for slope threshold crossing        -   Crossing of positive threshold a_(t)>T, is marked with +        -   Crossing of negative threshold a_(t)<−T is marked with −        -   Otherwise mark 0    -   Check for crossing pattern +−+− or −+−+, if such pattern (with        possible gaps within the provided signal, e.g., +0-+00-, up to        reasonable interval) appears, then report positive decision,        otherwise report negative decision    -   Not very sensitive to the off cycle duration.

FIG. 17 illustrates samples of positive and negative double tappinggestures accurately detected by the slope-estimation, time-domain basedtechnique described above. The technique is also referred to as theclassic detection technique. The technique provided correct decisions onall 25 positive and 25 negative examples of the double tap gesturedescribed previously. For each case (positive or negative sample), boththe computed slope and the gesture detection decision are plottedagainst time. A moving window is applied to generate these results.

Referring to FIG. 17, two examples are shown on the left and labeled as“Example for a single double-tapping.” Here, both examples are singledouble-tapping gestures. As shown, the time-domain based techniqueaccurately detected the double-tapping gesture. Two examples are shownin the middle and labeled as “Example of a no-tapping followed by twodouble-tapping.” Here both examples start as no tapping, followed by thedouble-tapping gesture performed twice in succession. Two examples areshown on the right of the figure and labeled as “Negative samples[without double-tapping].” Here both examples recorded received signalsduring which the user performed no double-tapping gesture.

FIG. 18 illustrates additional examples of “positive” double-tappingdetection using the time-domain technique (user performs double-tappinggesture). FIG. 19 illustrates additional examples of “negative”double-tapping detection using the same time-domain technique (user doesnot perform any double-tapping gesture).

According to various embodiments described above, gesture detection maybe performed on decimated signals in which millimeter wave signals forradar-based gesture detection are interspersed with millimeter wavesignals for data communications, i.e., network traffic. For example,double-tapping gesture detection is evaluated in real time records,e.g., 5 seconds of received radar signals. It is demonstrated that keycharacteristics of the gesture may be present in both full samples(non-decimated) as well as an interspersed (decimated) signal, e.g., 8msec “on” and 8 msec “off” millimeter wave radar signal for gesturedetection. Two categories of detection schemes—frequency-domain andtime-domain Both frequency and time-domain detection techniques areillustrated. Frequency-domain techniques include machine learning (ML)classifiers based on PCA-reduced spectrograms. Time-based techniquesinclude a phase detector scheme based on patterns of phase polarities.

Thus, improved techniques for gesture recognition using mm wave radarsignals produced by RF antennas compatible with 802.11 wi-fi protocolsbeen described. It will be appreciated that a number of alternativeconfigurations and fabrication techniques may be contemplated.

As used herein, a phrase referring to “at least one of” a list of itemsrefers to any combination of those items, including single members. Asan example, “at least one of: a, b, or c” is intended to cover: a, b, c,a-b, a-c, b-c, and a-b-c.

The various illustrative logics, logical blocks, modules, circuits andalgorithm processes described in connection with the implementationsdisclosed herein may be implemented as electronic hardware, computersoftware, or combinations of both. The interchangeability of hardwareand software has been described generally, in terms of functionality,and illustrated in the various illustrative components, blocks, modules,circuits and processes described above. Whether such functionality isimplemented in hardware or software depends upon the particularapplication and design constraints imposed on the overall system.

The hardware and data processing apparatus used to implement the variousillustrative logics, logical blocks, modules and circuits described inconnection with the aspects disclosed herein may be implemented orperformed with a general purpose single- or multi-chip processor, adigital signal processor (DSP), an application specific integratedcircuit (ASIC), a field programmable gate array (FPGA) or otherprogrammable logic device, discrete gate or transistor logic, discretehardware components, or any combination thereof designed to perform thefunctions described herein. A general purpose processor may be amicroprocessor or any conventional processor, controller,microcontroller, or state machine. A processor also may be implementedas a combination of computing devices, e.g., a combination of a DSP anda microprocessor, a plurality of microprocessors, one or moremicroprocessors in conjunction with a DSP core, or any other suchconfiguration. In some implementations, particular processes and methodsmay be performed by circuitry that is specific to a given function.

In one or more aspects, the functions described may be implemented inhardware, digital electronic circuitry, computer software, firmware,including the structures disclosed in this specification and theirstructural equivalents thereof, or in any combination thereof.Implementations of the subject matter described in this specificationalso can be implemented as one or more computer programs, i.e., one ormore modules of computer program instructions, encoded on a computerstorage media for execution by or to control the operation of dataprocessing apparatus.

If implemented in software, the functions may be stored on ortransmitted over as one or more instructions or code on acomputer-readable medium, such as a non-transitory medium. The processesof a method or algorithm disclosed herein may be implemented in aprocessor-executable software module which may reside on acomputer-readable medium. Computer-readable media include both computerstorage media and communication media including any medium that can beenabled to transfer a computer program from one place to another.Storage media may be any available media that may be accessed by acomputer. By way of example, and not limitation, non-transitory mediamay include RAM, ROM, EEPROM, CD-ROM or other optical disk storage,magnetic disk storage or other magnetic storage devices, or any othermedium that may be used to store desired program code in the form ofinstructions or data structures and that may be accessed by a computer.Also, any connection can be properly termed a computer-readable medium.Disk and disc, as used herein, includes compact disc (CD), laser disc,optical disc, digital versatile disc (DVD), floppy disk, and Blu-raydisc where disks usually reproduce data magnetically, while discsreproduce data optically with lasers. Combinations of the above shouldalso be included within the scope of computer-readable media.Additionally, the operations of a method or algorithm may reside as oneor any combination or set of codes and instructions on a machinereadable medium and computer-readable medium, which may be incorporatedinto a computer program product.

Various modifications to the implementations described in thisdisclosure may be readily apparent to those skilled in the art, and thegeneric principles defined herein may be applied to otherimplementations without departing from the spirit or scope of thisdisclosure. Thus, the claims are not intended to be limited to theimplementations shown herein, but are to be accorded the widest scopeconsistent with this disclosure, the principles and the novel featuresdisclosed herein. Additionally, as a person having ordinary skill in theart will readily appreciate, the terms “upper” and “lower”, “top” andbottom”, “front” and “back”, and “over”, “on”, “under” and “underlying”are sometimes used for ease of describing the figures and indicaterelative positions corresponding to the orientation of the figure on aproperly oriented page, and may not reflect the proper orientation ofthe device as implemented.

Certain features that are described in this specification in the contextof separate implementations also can be implemented in combination in asingle implementation. Conversely, various features that are describedin the context of a single implementation also can be implemented inmultiple implementations separately or in any suitable subcombination.Moreover, although features may be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed to achieve desirableresults. Further, the drawings may schematically depict one more exampleprocesses in the form of a flow diagram. However, other operations thatare not depicted can be incorporated in the example processes that areschematically illustrated. For example, one or more additionaloperations can be performed before, after, simultaneously, or betweenany of the illustrated operations. In certain circumstances,multitasking and parallel processing may be advantageous. Moreover, theseparation of various system components in the implementations describedabove should not be understood as requiring such separation in allimplementations, and it should be understood that the described programcomponents and systems can generally be integrated together in a singlesoftware product or packaged into multiple software products.Additionally, other implementations are within the scope of thefollowing claims. In some cases, the actions recited in the claims canbe performed in a different order and still achieve desirable results.

It will be understood that unless features in any of the particulardescribed implementations are expressly identified as incompatible withone another or the surrounding context implies that they are mutuallyexclusive and not readily combinable in a complementary and/orsupportive sense, the totality of this disclosure contemplates andenvisions that specific features of those complementary implementationsmay be selectively combined to provide one or more comprehensive, butslightly different, technical solutions. It will therefore be furtherappreciated that the above description has been given by way of exampleonly and that modifications in detail may be made within the scope ofthis disclosure.

Terms, “and” and “or” as used herein, may include a variety of meaningsthat also is expected to depend at least in part upon the context inwhich such terms are used. Typically, “or” if used to associate a list,such as A, B, or C, is intended to mean A, B, and C, here used in theinclusive sense, as well as A, B, or C, here used in the exclusivesense. In addition, the term “one or more” as used herein may be used todescribe any feature, structure, or characteristic in the singular ormay be used to describe some combination of features, structures, orcharacteristics. However, it should be noted that this is merely anillustrative example and claimed subject matter is not limited to thisexample. Furthermore, the term “at least one of” if used to associate alist, such as A, B, or C, can be interpreted to mean any combination ofA, B, and/or C, such as A, AB, AA, AAB, AABBCCC, etc.

What is claimed is:
 1. A method of gesture recognition comprising:performing gesture recognition and network data communications using anelectronic device, the electronic device having both a radar capabilityand a wireless communications capability based on millimeter wavesignals, the electronic device including at least one transmit antennaand at least one receive antenna that are operable in one or morefrequency ranges greater than 20 GHz, wherein the performing gesturerecognition includes: simultaneously operating of the at least onetransmit antenna and the at least one receive antenna so as to providethe radar capability, including receiving millimeter wave signals usingthe at least one receive antenna during a first portion of a time frame;detecting a presence and a motion of a reflective object by analyzing atleast phase of millimeter wave signals received by the at least onereceive antenna and resultant from reflection of signals transmitted bythe at least one transmit antenna by the reflective object; andoutputting a recognized gesture, wherein the performing network datacommunications includes: receiving millimeter wave signals using the atleast one receive antenna during a second portion of the time frame; anddemodulating and decoding the millimeter wave signals received duringthe second portion of the time frame to generate data bits, wherein thefirst portion of the time frame comprises a first plurality ofnon-contiguous time periods for receiving signals for gesturerecognition, wherein the second portion of the time frame comprises asecond plurality of non-contiguous time periods for receiving signalsfor network data communications, and wherein the first plurality ofnon-contiguous time periods of the time frame is interspersed with thesecond plurality of non-contiguous time periods of the time frame. 2.The method of claim 1, wherein millimeter wave signals received duringat least one of the first plurality of non-contiguous time periodscomprises a burst of pulses.
 3. The method of claim 2, wherein theanalyzing at least phase of the millimeter wave signals comprisescombining multiple pulses in the burst of pulses to generate a combinedchannel impulse response (CIR).
 4. The method of claim 3, where theanalyzing at least phase of the millimeter wave signals furthercomprises a frequency-based analysis, including: applying a transformoperation to the combined CIR to generate a spectrogram; providing thespectrogram to a trained machine learning (ML) classifier; and obtainingthe recognized gesture from the trained ML classifier.
 5. The method ofclaim 4, wherein the applying the transform operation comprisesperforming a Fast Fourier Transform (FFT) operation on the CIR togenerate the spectrogram.
 6. The method of claim 3, where the analyzingat least phase of millimeter wave signals further comprises a time-basedanalysis, including: generating an estimated slope reflecting a rate ofchange of phase based on the combined CIR; comparing the estimated slopeto a threshold; and determining a pattern of slope polarities based onthe comparing the estimated slope to the threshold; and generating therecognized gesture based on the pattern of the slope polarities.
 7. Themethod of claim 1, wherein the performing gesture recognition includesrecognizing a double tap gesture.
 8. The method of claim 1, wherein theat least one transmit antenna and at least one receive antenna arecompatible with one or both of IEEE 802.11ad and IEEE 802.11ay wi-fiprotocols.
 9. The method of claim 1, wherein each of the at least onetransmit antenna and the at least one receive comprises a plurality ofantenna elements.
 10. The method of claim 1, wherein the reflectiveobject is one or more of a hand or other appendage of a user, or a handheld object.
 11. The method of claim 1, wherein the signals transmittedby the at least one transmit antenna include two complementary Golaysequences used as two sequential radar pulses.
 12. The method of claim1, wherein one or both of the at least one transmit antenna and the atleast one receive antenna are operable in a 60 GHz band.
 13. The methodof claim 1, further comprising executing a graphical user interface(GUI) operation, responsive to the recognized gesture.
 14. An apparatuscomprising: a processor and an electronic device having both a radarcapability and a wireless communications capability based on millimeterwave signals, the electronic device including at least one transmitantenna and at least one receive antenna that are operable in one ormore frequency ranges greater than 20 GHz; wherein the processor isconfigured to perform gesture recognition with the electronic device by:simultaneously operating the at least one transmit antenna and the atleast one receive antenna so as to provide the radar capability,including receiving millimeter wave signals using the at least onereceive antenna during a first portion of a time frame; detecting apresence and a motion of a reflective object by analyzing at least phaseof millimeter wave signals received by the at least one receive antennaand resultant from reflection of signals transmitted by the at least onetransmit antenna and reflected by the reflective object; and outputtinga recognized gesture, wherein the processor is further configured toperform network data communications with the electronic device by:receiving millimeter wave signals using the at least one receive antennaduring a second portion of the time frame; and demodulating and decodingthe millimeter wave signals received during the second portion of thetime frame to generate data bits, wherein the first portion of the timeframe comprises a first plurality of non-contiguous time periods forreceiving signals for gesture recognition, wherein the second portion ofthe time frame comprises a second plurality of non-contiguous timeperiods for receiving signals for network data communications, andwherein the first plurality of non-contiguous time periods of the timeframe is interspersed with the second plurality of non-contiguous timeperiods of the time frame.
 15. The apparatus of claim 14, whereinmillimeter wave signals received during at least one of the firstplurality of non-contiguous time periods comprises a burst of pulses.16. The apparatus of claim 15, wherein the analyzing at least phase ofmillimeter wave signals comprises combining multiple pulses in the burstof pulses to generate a combined channel impulse response (CIR).
 17. Theapparatus of claim 16, where the analyzing at least phase of millimeterwave signals further comprises a frequency-based analysis, including:applying a transform operation to the combined CIR to generate aspectrogram; providing the spectrogram to a trained machine learning(ML) classifier; and obtaining the recognized gesture from the trainedML classifier.
 18. The apparatus of claim 17, wherein the applying thetransform operation comprises performing a Fast Fourier Transform (FFT)operation on the CIR to generate the spectrogram.
 19. The apparatus ofclaim 16, where the analyzing at least phase of millimeter wave signalsfurther comprises a time-based analysis, including: generating anestimated slope reflecting a rate of change of phase based on thecombined CIR; comparing the estimated slope to a threshold; anddetermining a pattern of slope polarities based on the comparing theestimated slope to the threshold; and generating the recognized gesturebased on the pattern of the slope polarities.
 20. The apparatus of claim14, wherein the performing gesture recognition includes recognizing adouble tap gesture.
 21. The apparatus of claim 14, wherein the at leastone transmit antenna and at least one receive antenna are compatiblewith one or both of IEEE 802.11ad and IEEE 802.1lay wi-fi protocols. 22.The apparatus of claim 14, wherein each of the at least one transmitantenna and the at least one receive antenna comprises a plurality ofantenna elements.
 23. The apparatus of claim 14, wherein the reflectiveobject is one or more of a hand or other appendage of a user, or a handheld object.
 24. The apparatus of claim 14, wherein the signalstransmitted by the at least one transmit antenna include twocomplementary Golay sequences used as two sequential radar pulses. 25.The apparatus of claim 14, wherein one or both of the at least onetransmit antenna and the at least one receive antenna are operable in a60 GHz band.
 26. The apparatus of claim 14, further comprising executinga graphical user interface (GUI) operation, responsive to the recognizedgesture.
 27. A non-transitory computer readable medium storing programcode to be executed by a processor, the program code comprisinginstructions configured to cause the processor to: perform gesturerecognition and network data communications using an electronic device,the electronic device having both a radar capability and a wirelesscommunications capability based on millimeter wave signals, theelectronic device including at least one transmit antenna and at leastone receive antenna that are operable in one or more frequency rangesgreater than 20 GHz, wherein the perform gesture recognition includes:simultaneously operating of the at least one transmit antenna and the atleast one receive antenna so as to provide the radar capability,including receiving millimeter wave signals using the at least onereceive antenna during a first portion of a time frame; detecting apresence and a motion of a reflective object by analyzing at least phaseof millimeter wave signals received by the at least one receive antennaand resultant from reflection of signals transmitted by the at least onetransmit antenna by the reflective object; and outputting a recognizedgesture, wherein the performing network data communications includes:receiving millimeter wave signals using the at least one receive antennaduring a second portion of the time frame; and demodulating and decodingthe millimeter wave signals received during the second portion of thetime frame to generate data bits, wherein the first portion of the timeframe comprises a first plurality of non-contiguous time periods forreceiving signals for gesture recognition, wherein the second portion ofthe time frame comprises a second plurality of non-contiguous timeperiods for receiving signals for network data communications, andwherein the first plurality of non-contiguous time periods of the timeframe is interspersed with the second plurality of non-contiguous timeperiods of the time frame.
 28. An apparatus comprising: a processor andan electronic device having both a radar capability and a wirelesscommunications capability based on millimeter wave signals, theelectronic device including at least one transmit antenna and at leastone receive antenna that are operable in one or more frequency rangesgreater than 20 GHz; means for performing gesture recognition with theelectronic device by: simultaneously operating the at least one transmitantenna and the at least one receive antenna so as to provide the radarcapability, including receiving millimeter wave signals using the atleast one receive antenna during a first portion of a time frame;detecting a presence and a motion of a reflective object by analyzing atleast phase of millimeter wave signals received by the at least onereceive antenna and resultant from reflection of signals transmitted bythe at least one transmit antenna by the reflective object; andoutputting a recognized gesture, and means for performing wirelesscommunications with the electronic device by: receiving millimeter wavesignals using the at least one receive antenna during a second portionof the time frame; and demodulating and decoding the millimeter wavesignals received during the second portion of the time frame to generatedata bits, wherein the first portion of the time frame comprises a firstplurality of non-contiguous time periods for receiving signals forgesture recognition, wherein the second portion of the time framecomprises a second plurality of non-contiguous time periods forreceiving signals for network data communications, and wherein the firstplurality of non-contiguous time periods of the time frame isinterspersed with the second plurality of non-contiguous time periods ofthe time frame.